Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

benchmarks: Improve benchmark for better precision and run time. #97

Merged
merged 1 commit into from
Jan 10, 2024
Merged

benchmarks: Improve benchmark for better precision and run time. #97

merged 1 commit into from
Jan 10, 2024

Conversation

jeanplevesque
Copy link
Member

@jeanplevesque jeanplevesque commented Jan 9, 2024 •
edited
Loading

GitHub Issue: #

Proposed Changes

  • Bug fix
  • Feature
  • Code style update (formatting)
  • Refactoring (no functional changes, no api changes)
  • Build or CI related changes
  • Documentation content changes
  • Other, please describe: Benchmarks

What is the current behavior?

  • Some benchmarks don't follow best practices from the BenchmarkDotNet documentation, which invalidates the results.
    • IterationSetup is used on iterations that last less than 100 ms.
  • The result table is bloated.
Method Job InvocationCount UnrollFactor Mean Error StdDev Median Gen0 Gen1 Gen2 Allocated
CreateViewModel DefaultJob Default 16 1.280 μs 0.0255 μs 0.0615 μs 1.268 μs 0.1488 0.1450 0.0019 936 B
CreateViewModel_WithExplicitName DefaultJob Default 16 1.193 μs 0.0236 μs 0.0419 μs 1.188 μs 0.1488 0.1469 - 936 B
ReadProperty_Unresolved Job-PVCBIR 1 1 2.810 μs 0.0599 μs 0.1376 μs 2.800 μs - - - 856 B
ReadProperty_Resolved Job-PVCBIR 1 1 1.515 μs 0.0342 μs 0.0941 μs 1.500 μs - - - 720 B
SetProperty_Unresolved Job-PVCBIR 1 1 19.909 μs 0.4667 μs 1.3389 μs 19.800 μs - - - 2592 B
SetProperty_Resolved Job-PVCBIR 1 1 1.831 μs 0.0401 μs 0.0763 μs 1.800 μs - - - 656 B
DisposeViewModel Job-PVCBIR 1 1 1.335 μs 0.0318 μs 0.0880 μs 1.300 μs - - - 600 B
SetProperty_Unresolved_WithListener Job-PVCBIR 1 1 21.604 μs 0.4888 μs 1.3627 μs 21.600 μs - - - 2616 B
SetProperty_Resolved_WithListener Job-PVCBIR 1 1 2.625 μs 0.0730 μs 0.2047 μs 2.600 μs - - - 680 B
DisposeViewModel_WithListener Job-PVCBIR 1 1 1.417 μs 0.0344 μs 0.0943 μs 1.400 μs - - - 600 B

What is the new behavior?

  • Commits with the benchmarks conventional commit type are now allowed.
  • IterationSetup is used on iterations that last more than 100 ms in conjuction with OperationsPerInvoke.
  • Benchmarks about the ViewModelBase class and the extension methods on IViewModel are now better separated.
  • The results table is lean.
Method Mean Gen0 Gen1 Allocated
CreateViewModel 623.60 ns 0.0858 0.0849 544 B
CreateViewModel_WithExplicitName 587.77 ns 0.0858 0.0849 544 B
DisposeViewModel 62.59 ns - - -
GetFromValue_Unresolved 427.69 ns 0.0405 0.0401 256 B
GetFromObservable_Unresolved 544.88 ns 0.0477 0.0467 304 B
GetFromValue_Resolved 46.09 ns 0.0191 - 120 B
GetFromObservable_Resolved 52.72 ns 0.0204 - 128 B
Set_Unresolved 1,210.68 ns 0.1106 0.0286 695 B
Set_Resolved 30.67 ns 0.0038 - 24 B

Impact on version

  • Major (Public API was modified.)
    • Public constructs (class, struct, delegate, enum, etc.) were removed or renamed.
    • Public members were removed or renamed.
    • Public method signatures were changed or renamed.
  • Minor (Public API was extended.)
    • Public constructs, members, or overloads were added.
  • Patch (Public API was unchanged.)
    • A bug in behavior was fixed.
    • Documentation was changed.
  • None (The library is unchanged.)
    • Only code under the build folder was changed.
    • Only code under the .github folder was changed.
    • Only code in the Benchmarks project was changed.

Checklist

Please check that your PR fulfills the following requirements:

  • Documentation has been added/updated.
  • Automated Unit / Integration tests for the changes have been added/updated.
  • Updated BREAKING_CHANGES.md (if you introduced a breaking change).
  • Your conventional commits are aligned with the Impact on version section.

Other information

@jeanplevesque jeanplevesque marked this pull request as ready for review January 9, 2024 22:35
@jeanplevesque jeanplevesque requested a review from a team January 9, 2024 22:36
@jeanplevesque jeanplevesque force-pushed the dev/jpl/benchmarks branch 2 times, most recently from 09c3da4 to 652d57b Compare January 9, 2024 23:01
@jeanplevesque jeanplevesque merged commit 80a56ae into main Jan 10, 2024
4 checks passed
@jeanplevesque jeanplevesque deleted the dev/jpl/benchmarks branch January 10, 2024 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@takla21 takla21 takla21 approved these changes

@lamonfly lamonfly lamonfly approved these changes

@Soap-141 Soap-141 Soap-141 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jeanplevesque @takla21 @lamonfly @Soap-141